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Abstract 

Descriptions  of  complex  concepts  often  use  exam¬ 
ples  for  illustrating  various  points.  This  paper  dis¬ 
cusses  the  issues  that  arise  in  generating  complex 
descriptions  in  tutorial  contexts.  Although  some  tu¬ 
torial  systems  have  used  examples  in  explanations, 
they  have  rarely  been  considered  as  an  integral  part 
of  the  complete  explanation  -  they  have  usually 
been  merely  supportive  devices  -  and  inserted  in 
the  explanations  without  any  representation  in  the 
system  of  how  the  examples  relate  to  and  comple¬ 
ment  the  textual  explanations  that  accompany  the 
examples.  This  can  lead  to  presentations  that  are  at 
best,  weakly  coherent,  and  at  worst,  confusing  and 
mis-leading  for  the  learner. 

In  this  paper,  we  consider  the  genwation  of  exam¬ 
ples  as  an  integral  part  of  the  overall  process  of 
generation,  resulting  in  examples  and  text  that  are 
smoothly  integrated  and  complement  each  other. 
We  address  the  requirements  of  a  system  capable 
of  this,  and  present  a  framework  in  which  it  is  pos¬ 
sible  to  generate  examples  as  an  integral  part  of  a 
description.  We  then  show  how  techniques  devel¬ 
oped  in  Natural  Language  Generation  can  be  used 
to  build  such  a  framework. 


1  Introduction 

It  has  long  been  known  that  examples  are  very  useful  in 
communication  -  especially  in  explanations  and  instruction. 
New  ideas,  concepts  or  terms  are  conveyed  with  greater  ease 
and  ciaiity,  if  the  descriptions  are  accompanied  by  appropri¬ 
ate  examples  (e.g.,  [Houtz  el  ai,  1973;  MacLachlan.  1986; 
Wrolli,  1991;  Reder  et  al.,  1986;  Tennyson  and  Park,  1980]). 
People  like  examples  because  examples  tend  to  put  absuact, 
theoretical  information  into  concrete  terms  they  can  under¬ 
stand. 

We  gratefully  acknowledge  the  support  of  NASA -Ames 
grant  NCC  2-520  and  DARPA  contract  DABT63-9I-C-0025. 
The  autfaon  may  he  contacted  through  electronic  mail  at; 
(MrrTAL.PAiUS)  OISI.EDU 


Explanation  capabilities  are  becoming  increasingly  impor¬ 
tant  nowadays,  especially  as  domain  models  become  larger 
and  more  specialized.  One  requirement  in  such  systems  is 
the  ability  to  explain  complex  objects,  relations  or  processes 
that  are  represented  in  the  system.  Advances  in  natural  lan¬ 
guage  generation  and  research  in  user  modelling  have  resulted 
in  impressive  descriptions  being  produced  by  such  systems. 
However,  these  systems  have  not,  for  the  most  part,  con¬ 
centrated  on  the  issue  of  generating  examples  as  a  part  of 
the  overall  description.  While  examples  can,  in  some  cases, 
be  retrieved  from  a  pre-dehned  example-base'  and  added  to 
the  description,  using  examples  effectively,  as  an  important 
and  a  complementary  part  of  the  overall  description,  requires 
the  system  to  reason  with  the  constraints  introduced  by  both 
the  textual  explanation,  as  well  as  the  examples,  in  making 
decisions  during  the  generation  process. 

There  are  many  issues  that  must  be  considered  in  selecting 
and  presenting  examples  -  in  this  paper,  we  shall  briefly  high¬ 
light  some  of  these  issues.  We  view  example  generation  as 
an  integral  part  of  generating  descriptions,  because  examples 
affect  not  only  other  examples  that  follow,  but  also  the  sur¬ 
rounding  text.  We  describe  how  a  text  generation  system  that 
plans  text  in  terms  of  both  intentional  and  rhetorical  goals, 
can  be  suuctured  to  plan  utterances  that  can  include  examples 
in  an  integrated  and  coherent  fashion.  Some  of  the  issues 
addressed  in  this  regard  are  equally  important  in  the  plan¬ 
ning  and  presenution  of  other  explanatory  devices  -  such  as 
diagrams,  pictures  and  analogies. 


2  Previous  Work  and  Unaddressed  Issues 

Most  previous  approaches  to  the  use  of  examples  in  gen¬ 
erating  descriptions  and  explanations  focused  on  the  issue 
of  finding  useful  examples.  Rissland's  (1981)  CEO  sys¬ 
tem  ,  for  instance,  investigated  issues  of  retrieval  versus  con¬ 
struction  of  examples;  Rissland  and  Ashley's  (1986)  HYPO 
system  retrieved  examples  and  investigate  techniques  for 
modifying  them  along  multiple  dimensions  to  fit  required 
specifications;  Suther's  example  generator  [Suthers  and  Riss¬ 
land,  19881  is  also  sinular  to  CEG,  ar.l  invcsvigalcd  effi¬ 
ciency  ot  search  techniques  in  finding  examples  to  modify. 
Later  work  by  Woolf  and  her  colleagues  focused  on  de¬ 
sign  issues  of  tutoring  systems,  including  determination  of 
when  examples  are  necessary  [Woolf  and  McDonald,  1984; 
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Woolf  and  Murray,  19871.  However,  it  did  not  address  is¬ 
sues  of  discourse  generation  and  the  integration  of  text  with 
examples. 

Our  work  builds  upon  these  and  other  studies  (e.g.,  [Reiser 
ei  aL,  1985:  Rlssland  «  al.,  1984;  Woolf  et  aL,  19881)  to 
study  bow  to  provide  appropriate,  well-structured  and  coher¬ 
ent  examples  in  the  context  of  the  surrounding  text.  It  has  been 
shown  previously  that  presentation  of  descriptions  without  the 
use  of  examples  is  not  effective,  perhaps  because  the  use  of 
definitions  on  their  own  may  lead  to  a  learner  merely  memo¬ 
rizing  a  string  of  verbal  associations  [Klausmeier,  19761.  The 
converse  -  presentation  of  examples  aloue  -  has  also  been 
shown  previously  to  be  less  eff^ective  than  the  use  of  both  ex¬ 
amples  and  descriptions;  the  use  of  both  almost  doubled  user 
comprdiension  in  certain  cases  (e.g.,  [Charney  et  al.,  1988; 
Feldman,  1972;  Ftidman  and  Klausmeier,  1974;  Gilling¬ 
ham,  19^;  Klausmeier,  1976;  Merrill  and  Tennyson,  1978; 
Reder  et  al,  19861).  However,  text  and  examples  that  are  not 
well  integrated  can  cause  greater  confusion  than  either  one 
alone  (e.g.,  [Ward  and  Sweller,  19901).  Thus  it  is  clear  that  if 
descriptions  are  to  make  use  of  examples  effectively,  both  the 
descriptions  and  the  examples  must  be  integrated  with  each 
other  in  a  coherent  and  complementary  fashion.  Furthermore, 
there  is  often  a  lot  of  implicit  information  in  the  sequence  of 
presentation  of  the  examples.  The  system  should  be  capa¬ 
ble  of  representing  this  information  to  structure  the  sequence 
correctly. 

There  are  many  points  that  must  be  considered  by  any  sys¬ 
tem  that  attempu  to  generate  effecti  vedescriptions  of  complex 
concepts; 

1.  What  aspects  of  information  should  be  exemplified?  A 
system  is  likely  to  have  detailed  knowledge  about  the 
concq)t.  It  typically  needs  to  select  some  aspects  to 
present  to  the  user.  (This  issue  has  often  been  raised  in 
natural  language  generation.) 

2.  When  should  a  description  include  examples?  A  few 
researchers  have  started  to  look  at  this  issue  [Woolf  and 
McDonald,  1984;  Woolf  and  Murray,  1987;  Woolfeta/., 
1988],  although  much  work  still  remains  to  be  done. 

3.  How  is  a  suitable  example  found?  Is  it  retrieved  from 
a  pre-defined  knowledge  base  and  modified  to  meet  cur¬ 
rent  specifications  (as  in  (Rissland  et  al,  1984]),  or  is  it 
constructed  (as  in  [Rissland,  1981])? 

4.  How  should  the  example  be  positioned  with  respect  to 
the  surrounding  text?  Should  the  example  be  within  the 
text,  before  it,  or  etfter  it? 

5.  Should  the  system  use  one  example,  or  multiple  exam¬ 
ples?  If  more  than  one  example  is  used,  how  many 
should  be  used,  and  what  information  should  each  one 
convey? 

6.  If  multiple  examples  are  to  be  presented,  how  is  the  order 
of  presentation  to  be  determined? 

7.  What  information  should  be  included  in  the  prompts' 
and  how  can  they  be  generated? 

'  'Prompts'  are  atlentioD  focusing  devices  such  as  arrows,  marks, 
or  even  additional  text  associated  with  examples  [Engelmann  and 
ermine,  1982]. 


A  list  always  begins  with  a  left  parenthesis.  Then  come  zero 
or  more  pieces  of  data  (called  the  elements  of  a  list)  and  a 
right  parenthesis.  Some  examples  of  lists  are: 

(AAROVARK)  ;;;anatom 

(RED  YELLOW  GREEN  BLUE)  ;;;  many  atoms 
(2  3  5  11  19)  ;;;  numbers 

(3  FRENCH  FRIES)  ;;;  atoms  &  numbers 

A  list  may  contain  other  lists  as  elements.  Given  the  three 
lists: 

(BLUE  SKY)  (GREEN  GRASS)  (BROWN  EARTH) 
we  can  make  a  list  by  combining  them  all  with  a  parentheses. 

((BLUE  SKY)  (GREEN  GRASS)  (BROWN  EARTH)) 

Figure  1:  A  description  of  the  object  LIST  using  examples 
(From  [Touretzky,  1984],  p.35) 


8.  How  is  lexical  cohesion  maintained  between  the  exam¬ 
ple  and  the  text?  The  text  and  the  example(s)  should 
probably  use  the  same  lexical  items  to  refer  to  the  same 
concepts. 

9.  We  already  know  from  work  in  natural  language  genera¬ 
tion  that  a  description  is  affected  by  issues  such  as  user- 
type,  text-type,  etc.  How  is  the  generation  of  examples 
(when  they  should  be  included,  the  type  of  information 
that  they  illustrate,  and  the  number  of  examples)  within 
a  text  affected  by: 

•  the  prospective  audience-type  (naive  vs.  advanced, 
for  instance), 

•  the  knowledge-type  (concepts  vs.  relations  vs.  pro¬ 
cesses), 

•  the  text-type  (tutorial  vs.  reference  vs.  report,  etc), 
and 

•  the  dialogue  context? 

While  each  of  these  issues  needs  to  be  addressed  in  a  prac¬ 
tical  system,  we  shall  only  discuss  some  of  them  here:  the 
issues  of  positioning  the  example  within  a  text,  the  num¬ 
ber  of  examples  to  be  presented,  and  their  order.  (Issues 
#1.  #2  and  #3  have  already  been  studied  to  some  extent,  by 
other  researchers,  for  e.g.,  [Paris,  1987;  Woolf  et  al.,  1988; 
Rissland  et  al,  1984;  Rissland  and  Ashley,  1986;  Rissland, 
1983]).  We  now  discuss  in  more  detail,  the  points  we  are  con¬ 
cerned  with.  We  illustrate  each  point  with  the  description  in 
Rgure  1 .  We  are  not  yet  (for  this  paper)  addressing  the  issues 
of  user-type,  the  text-type  and  the  dialogue  context,  though 
they  can  all  affect  the  generation  of  examples.  We  take  as 
our  initial  context  the  generaion  of  a  description  in  a  tutorial 
fashion,  for  a  naive  user  and  as  a  ‘one-shot’  response. 

3  Integrating  Exanip!t;s  in  De;^  Iption.< 

As  mentioned  previously,  a  number  of  studies  have  shown 
the  need  for  examples  to  illustrate  descriptions  and  dehnitions, 
as  well  as  a  need  for  explanations  to  complement  the  examples 


presented.  Presentation  of  either  on  their  own  is  not  as  use^l 
an  approach  as  one  that  combines  both  of  them  together. 

3.1  Positkmii^  the  Example  and  the  Description: 

Should  the  example  be  placed  before,  within  or  after  the  ac¬ 
companying  text?  This  is  an  important  issue,  as  it  has  been 
reported  that  there  are  significant  differences  that  can  result 
from  the  placement  of  examples  before  and  after  the  explana¬ 
tion  [Feldman,  1972}. 

Our  analyses  of  different  instructional  matoials  reveal  that 
examples  usually  tend  to  follow  the  description  of  a  concqit  in 
terms  of  its  critical  attributes.  Critical  attributes  are  attributes 
that  are  definitional  -  the  absence  of  any  of  these  attributes 
causes  an  instance  to  not  be  an  example  of  a  concept.  In 
the  lisp  domain,  for  example,  a  LIST  has  parentheses  as  its 
critical  attributes;  the  elements  of  the  list  itself  are  not.  The 
examples  can  then  be  followed  by  text  which  elaborates  on 
features  in  the  examples,  unless  prompts  are  included  with 
the  examples.  If  more  than  one  example  needs  to  be  pre¬ 
sented.  the  example  is  usually  pl^xd  separately  from  the  text 
(rather  than  within  it).  Otherwise,  the  example  is  integrated 
within  the  text,  as  in:  An  azaaple  of  a  string  is  "The 
quick  brosn  fox".  Following  the  examples,  the  descrip¬ 
tion  continues  with  otho^  attributes  of  the  concept,  possibly 
accompanied  by  further  examples. 

Sometimes,  examples  are  used  as  elaborations  for  certain 
points  which  might  otherwise  have  been  elaborated  upon  in 
the  text.  For  instance,  in  Figure  1.  the  LIST  could  have  been 
described  as  "A  list  always  begins  with  a  left  parenthesis. 
Then  come  zero  or  more  pieces  of  data,  which  can  be  either 
symbols,  numbers,  or  combinations  of  symbols  and  numbers. 
followed  by  a  right  parenthesis.”  Instead,  the  elaboration  on 
the  data  types  is  embodied  in  the  information  present  in  the 
examples.  The  first  set  of  examples  in  Figure  1  have  prompts 
associated  with  them,  highlighting  features  (number  and  type 
information)  about  the  examples.  Following  these  examples, 
some  text  elaborates  on  the  fact  that  the  elements  of  a  list  can 
also  be  lists.  Further  examples  of  lists  are  used  to  show  how 
these  can  be  combined  to  form  another  list. 

3J  Providing  the  Appropriate  Number  of  Examples 

Studies  have  indicated  that  information  transfer  is  maximized 
when  the  learner  has  to  concentrate  on  as  few  features  as 
possible  [Ward  and  Sweller,  1990].  This  implies  that  teach¬ 
ing  a  concqit  is  most  effectively  done  one  feature  at  a  time. 
This  has  important  implications  for  example  generation:  it 
indicates  that  examples  should  try  and  convey  one  point  at 
a  time,  especially  if  the  examples  are  meant  to  teach  a  new 
concqit.  Thus,  should  the  concept  have  a  numbCT  of  different 
features,  a  number  of  examples  are  likely  to  be  required,  one 
(or  a  set  of)  examples  for  each  feature.  This  is  also  supported 
by  experiments  on  differences  in  learning  arising  from  using 
different  numbers  of  examples  [Clark,  1971;  Feldman,  1972; 
Klausmeier  and  F^droan,  197S;  Markle  and  Tiemann,  1969]. 

This  is  illustrated  in  Figure  1,  in  which  each  example  high¬ 
lights  one  feature  of  USTs:  that  the  data  can  be  a  single 
symbol,  a  number  of  symbols,  numbers,  etc.  Contrast  those 


examples,  with  a  single  example  which  summarizes  most  of 
the  features  a  LIST  can  have,  as  given  below: 

(FORKAT  T  "*  A'  A*  A"  '«bcd*r  1234S6 
*(abc  (123  ("ab")))  (  )) 

It  is  important  that  the  system  generate  an  appropriate  num¬ 
ber  of  examples,  each  emphasizing  certain  selected  features. 

3  J  Ordering  the  Examples 

Given  a  number  of  examples  to  present,  the  sequencing  is  also 
an  important  matter,  because  examples  often  build  upon  each 
other.  Furthermore,  the  difference  baween  two  adjacent  ex¬ 
amples  is  signihcani,  as  proper  sequencing  can  be  a  very  pow¬ 
erful  means  of  focusing  the  hearer’s  attention  (e.g..  (Feldman, 
1972;Houtz«fl/.,  1973;  Klausmeier  era/.,  1974;  Litchfield  er 
at,  1990;  Markle  and  Tiemann,  1969;  Tennyson  et  al.,  1975; 
Tennyson  and  Tennyson,  1975]).  Consider  the  sequence  of 
examples  on  LISTs  in  Figure  1:  the  first  two  examples  focus 
attention  on  the /lum/vr  of  elements  in  a  LIST  -  they  highlight 
the  fact  that  a  LIST  can  have  any  numbo-  of  elements  in  it:  the 
second  and  third  ones  illustrate  that  symbols  are  not  always 
required  in  a  LIST  -  a  LIST  can  ^so  be  made  up  of  numbers; 
the  fourth  example  contrasts  with  the  third,  and  illustrates  the 
point  that  a  LIST  need  not  have  elements  of  just  one  type  - 
both  numbers  as  well  as  symbols  can  be  in  a  LIST  at  the  same 
time. 

It  has  also  been  shown  that  presenting  easily  understood 
examples  before  presenting  difficult^  examples  has  a  signif¬ 
icant  beneficial  effect  on  learner  comprehension  [Gamine, 
19801.  Ordering  is  thus  important  -  it  is  worth  noting  that  the 
linguistic  notion  of  the  maxim  of  end-weight  [Giora,  1988; 
Werth,  1984],  also  dictates  that  difficult  and  new  items  should 
be  mentioned  after  easier  and  known  pieces  of  information; 
since  there  is  a  direct  correlation  between  the  description  and 
the  examples,  this  maxim  offers  additional  motivation  for  a 
sequencing  of  the  examples  from  easy  to  difficult.  Possible 
orderings  may  also  depend  upon  factors  such  as  the  type  of 
concept  being  communicated  (whether  for  instance,  it  is  a  dis¬ 
junctive  or  a  conjunctive  concept)  or  whether  it  is  a  relation. 
For  example,  in  Figure  1 ,  the  order  of  examples  is  determined 
both  by  the  order  in  which  features  are  mentioned  (“zero  or 
more”  and  “pieces  of  data”),  and  the  complexity  of  exam¬ 
ples  within  each  grouping  (symbols,  followed  by  numbers, 
followed  by  combinations  of  symbols  and  numbas). 

3.4  Generating  Prompts  for  the  Examples 

Instructional  materials  that  include  examples  often  have  tag 
information  associated  with  each  example.  This  is  often  re¬ 
ferred  to  as  “prompting”  information  in  educational  literature 
(e.g.,  [Engelmann  and  Gamine,  1982]).  Prompts  help  focus 
attention  on  the  feature  being  illustrated.  They  often  replace 
long,  detailed  explanations,  and  therefore  play  a  role  similar 
to  the  one  of  explanation  of  the  examples.  However,  as  they 

^The  terms  'easy'  and  'difficult'  are  difficult  to  specify,  and  are 
usually  highly  domain  specific  -  in  the  case  of  LISP,  for  instance, 
one  measure  of  difficulty  is  the  number  of  different  grammatical 
productions  that  would  be  required  to  parse  the  construct. 


occur  in  the  s^ne  sequence  as  the  examples,  they  capture  the 
change  in  the  examples  very  efficiently.  Consider  the  example 
sequence  in  Figure  1 .  In  this  case,  the  prompts  (tags  such  as 
“List  of  Symbols"  and  “List  of  combined  Symbols  and  Num¬ 
bers”)  cause  the  learner  to  focus  on  the  feature  that  is  being 
highlighted  by  the  exanqiies. 


3,5  Maintaining  Lexical  Cohesion  between  the  Text  and 
the  Examples 

In  any  given  domain,  there  are  likely  to  be  a  number  of  terms 
available  for  a  particular  concept.  It  is  important  that  the  sys¬ 
tem  use  consistent  terminology  throughout  the  description; 
both  in  the  definition,  as  well  as  in  the  examples.  Consider 
for  instance,  the  description  in  Figure  1.  Both  the  examples 
and  the  definition  use  the  term  ‘list’,  although  the  terms  list, 
s-expnssion  and  (sometinws)/orwj  are  interchangeable.  Dif¬ 
ference  in  terminology  can  result  in  confusing  messages  being 
communicated  to  the  learner  (e.g.,  [Feldman  and  Klausmeier, 
1974]).  This  issue  becomes  especially  important  in  cases 
where  the  examples  are  retrieved,  as  the  terms  used  in  the  ex¬ 
ample  may  be  different  from  the  terms  used  in  other  examples, 
or  the  definition.  The  construction  of  the  textual  definition  and 
the  examples  must  thus  be  done  in  a  coordinued  and  cooper¬ 
ative  manner. 


3.6  The  Knowledge-iypc  and  its  Effect  on  Descriptions 

It  has  been  obsCTved  that  the  type  of  the  knowledge  com¬ 
municated  (concept  vs.  relation  vs.  a  process)  has  im¬ 
portant  implications  for  the  manner  in  which  this  commu¬ 
nication  takes  place  (e.g.,  [Bruner,  1966;  Engel  mann  and 
Camine,  1982]).  Not  only  is  the  information  different,  but 
the  type  of  examples  and  their  order  of  presentation  is  af¬ 
fected.  This  is  b^ause,  if,  for  instance,  the  system  needs 
to  present  information  on  a  relation,  it  must  first  make  sure 
that  the  concepts  between  which  the  relation  holds  are  un¬ 
derstood  by  the  bearer  -  this  may  result  in  other  examples 
of  the  concepts  being  presented  before  examples  of  the  rela¬ 
tion  can  be  presented.  The  order  is  usually  different  too  and 
there  are  no  negative^  examples  of  relations  presented  in  ini¬ 
tial  teaching  sequences  (e.g.,  [Engelmann  and  Camine,  1982; 
Bruner,  1966]). 

In  this  section,  we  have  described  in  somewhat  greater 
detail,  a  few  of  the  issues  that  we  identified  in  Section  2.  In 
the  following  section,  we  describe  a  framework  for  generation 
that  addresses  some  of  the  above  issues.  We  should  mention 
that  our  system  has  only  a  simple  user-model,  and  in  the 
description,  we  shall  not  discuss  other  aspects  of  the  system 
such  as  how  dialogue  is  handled  [Moore,  1 989a],  the  effect  of 
the  text-type,  etc.  The  description  is  meant  to  convey  a  flavor 
of  how  the  system  processes  a  goal  to  describe  concepts  and 
uses  examples  to  help  achieve  its  goal. 

^‘Positive’  examples  are  instances  of  the  concept  they  illustrate; 
‘negative'  examples  are  those  which  are  not  instances  of  the  concept 
being  described. 


Figure  2:  A  block  diagram  of  the  overall  system. 


4  A  Framework  to  Generate  Descriptions 
with  Examples 

Using  techniques  developed  in  natural  language  generation, 
we  are  working  on  a  framework  within  which  it  is  pos¬ 
sible  to  integrate  examples  in  a  description.  This  frame¬ 
work  will  also  enable  us  to  investigate  and  test  more  care¬ 
fully  the  issues  raised  in  Section  2,  incorporating  and  build¬ 
ing  upon  the  work  of  other  researchers  (e.g.,  (Paris,  1991b; 
Woolfe/a/..  1988;Risslandeffl/.,  1984;Rissland«a/.,  1984; 
Rissland  et  ai,  1984]).  Our  cunent  framework  implements 
the  generation  of  examples  within  a  text-generation  system 
by  explicitly  posting  the  goals  of  providing  examples.  Our 
system  uses  a  planning  mechanism:  given  a  top  level  commu¬ 
nicative  goal  (such  as  (DESCRIBE  LIST) ),  the  system  finds 
plans  capable  of  achieving  this  goal.  Plans  typically  post 
further  sub-goals  to  be  satisfied,  and  planning  continues  un¬ 
til  primitive  speech  acts  -  i.e.,  directly  realizable  in  English 
-  are  achieved.  The  result  of  the  planning  process  is  a  dis¬ 
course  uee,  where  the  nodes  represent  goals  at  various  levels 
of  abstraction  (with  the  root  being  the  initial  goal,  and  the 
leaves  representing  primitive  realization  statements,  such  as 
(INFORM  . . . )  statements.  In  the  discourse  tree,  the  dis¬ 
course  goals  are  related  through  coherence  relations.  This 
tree  is  then  passed  to  a  grammar  interface  which  converts  it 
into  a  set  of  inputs  suitable  for  input  to  a  natural  language 
generation  system  (Penman  [Mann,  1983]).  A  block  diagram 
of  the  system  is  shown  in  Figure  2. 

Plan  operators  can  be  seen  as  small  schemas  (scripts)  which 
describe  how  to  achieve  a  goal;  they  are  designed  by  study¬ 
ing  natural  language  texts  and  transcripts.  They  include 
conditions  for  their  applicability.  These  conditions  can  re¬ 
fer  resources  like  the  system  knowledge  base  (KB),  the  user 
model,  or  the  context  (including  the  dial  ogue  context).  A  com¬ 
plete  description  of  the  generation  system  is  beyond  the  scope 
of  this  paper  -  see  [Moore  and  Paris,  1992;  Moore,  1989b: 
Moore  and  Paris,  1991;  Paris,  1991a;  Moore  and  Paris,  1989] 
for  more  details. 

We  are  adding  an  example  generator  to  this  generation  sys¬ 
tem.  Examples  are  generated  by  explicitly  posting  a  goal 


within  the  text  planning  system:  i.e.,  some  of  the  plan  oper¬ 
ators  used  in  the  system  include  the  generation  of  examples 
as  one  of  their  stq)s,  when  applicable.  This  ensures  that  the 
examples  embody  specific  information  that  either  illustrates 
or  complements  the  information  in  the  accompanying  textual 
description.  It  is  clear  that  there  are  additional  constraints  (for 
e.g.,  the  user  model,  text  type,  dialogue  context,  etc)  that  will 
be  needed  in  any  comprehensive  implementation  of  exam¬ 
ple  generation,  but  we  shall  investigate  those  issues  in  future 
work.  These  additional  sources  of  knowledge  can  be  currently 
added  to  the  system  by  incorporating  additional  constraints  in 
the  plan  operators  which  reference  these  resources.  Thus,  ex¬ 
perimenting  with  different  sources  in  an  effort  to  study  their 
effects  is  not  very  difficult. 

The  number  of  examples  that  the  system  needs  to  present 
is  determined  by  an  analysis  of  the  features  that  need  to  be 
illustrated.  These  features  dq)end  on  the  representation  of 
the  concept  in  the  knowledge  base  anJ  the  user  model.  Not 
all  the  features  illustrated  in  the  examples  may  be  actually 
mentioned  in  the  text.  This  is  because  the  desaiption  may 
actually  leave  the  elaboration  up  to  the  examples  rather  than 
doing  it  in  the  text.  In  Figure  1 ,  for  instance,  the  different  data 
types  (numbers,  symbols  or  combinations  of  both)  that  may 
form  the  elements  of  a  list  are  not  mentioned  in  the  text,  but  are 
illustrated  through  examples.  The  user  model  influences  the 
choice  of  features  to  be  presented.  The  number  of  examples 
is  directly  proportional  to  the  number  of  features  -  in  case  of 
the  naive  user,  there  is  usually  one  example  per  feature.  In 
our  framework,  the  featiues  to  be  presented  are  determined 
based  on  the  domain  model  and  a  primitive  categorization  of 
the  usCT  (naive  vs.  advanced). 

The  order  of  presentation  of  examples  is  dependent  mainly 
upon  the  order  of  the  features  being  mentioned  in  the  text. 
In  case  the  text  does  not  explicitly  mention  the  features  (as 
in  Figure  1,  where  the  different  dau  types  are  not  mentioned 
in  the  text),  the  system  orders  them  in  increasing  complexity. 
Since  the  ordering  in  the  text  is  in  an  increasing  order  of 
complexity  too  (the  maxim  of  end- weight),  the  least  complex 
examples  are  presented  first.  We  have  devised  domain  specific 
measures  of  complexity.  In  the  case  of  LISP  for  instance,  the 
complexity  of  a  structure  is  measured  in  terms  of  the  number 
of  difierent  productions  that  would  need  to  be  invoked  to  parse 
the  example. 

The  system  maintains  lexical  cohesion  by  replacing  all  oc¬ 
currences  of  equivalent  terms  with  one  uniform  term.  This 
is  done  as  the  last  step  in  the  discourse  tree,  before  it  is  used 
as  input  to  the  language  generator.  Tho-e  are  clearly  more 
issues  to  be  studied  to  obtain  lexical  cohesion,  but  this  indi¬ 
cates  our  framework’s  ability  to  at  least  ensure  a  consistent 
use  of  vocabulary.  Our  framework  is  thus  centered  around 
a  text-planner  that  generates  text  and  posts  explicit  goals  to 
generate  examples  that  will  be  included  in  the  description. 
Plans  also  indicate  how  and  when  to  generate  the  prompt 
information.  By  appropriately  modifying  the  consuaints  on 
each  plan-operator,  we  can  investigate  the  effects  of  differ¬ 
ent  resources  in  the  framework.  In  the  following  section, 
we  shall  illustrate  the  working  of  the  system  by  generating  a 
description  similar  to  the  one  in  Figure  1 . 


4.1  A  Truce  of  the  System 

The  system  initially  begins  with  the  top-level  goal  being  given 
as  (DESCRIBE  LIST) .  The  text  planna  searches  for  applica¬ 
ble  plan  operators  in  its  plan-library,  and  it  picks  one  based  on 
the  applicable  consfraints  such  as  the  user  model  (inuoduc- 
tory),  the  knowledge  type  (concept),  the  text  type  (scientific), 
etc.  The  user  model  restricts  the  choice  of  the  features  in 
this  case  (naive  user)  to  syntactic  ones.  The  main  features  of 
LIST  are  retrieved,  and  two  subgoals  are  posted;  one  to  list 
the  critical  features  (the  left  parenthesis,  the  data  elements  and 
the  right  parenthesis),  and  another  to  elaborate  upon  them. 

At  this  point,  the  discourse  tree  has  only  two  nodes;  the 
initial  node  of  (DESCRIBE  LIST)  -  namely  LIST-M4I1- 
-FERTURES  and  DESCRIBE-FERTURES.  linked  by  a  rhetorical 
relation,  ELRBORRTE.^ 

The  text-planner  now  has  these  two  goals  to  expand; 
LIST-MRII-FERTURES 
DESCRIBE-FERTURES 

The  planner  searches  for  appropriate  operators  to  satisfy  these 
goals.  The  plan  operator  to  describe  a  list  of  features  indicates 
that  the  features  should  be  mentioned  in  a  sequence.  Three 
goals  are  appropriately  posted  at  this  point.  These  goals  re¬ 
sult  in  the  planner  generating  a  plan  for  the  first  sentence  in 
Figure  1.  The  other  sub-goal  of  DESCRIBE-LIST  also  causes 
three  goals  to  be  posted  for  describing  each  of  the  critical 
features.  Since  two  of  these  are  for  elaborating  upon  the 
parentheses,  they  are  not  expanded  because  no  further  infor¬ 
mation  is  available.  A  skeleton  of  the  resulting  text  plan  is 
shown  in  Figure  3. 

The  system  now  attempts  to  satisfy  the  goal 
DESCRIBE-DRTR-ELEMENTS  by  finding  an  appropriate  plan. 
Data  elements  can  be  of  three  types:  numbers,  symbols,  or 
lists.  The  system  can  either  communicate  this  information 
by  realizing  an  appropriate  sentence,  or  thrcugh  examples  (or 
both).  The  text  type  and  user  model  consfraints  cause  the 
system  to  pick  examples.  It  generates  two  goals  for  the  two 
dimensions  in  which  the  data  elements  can  vary  in:  the  num¬ 
ber  and  the  type.  The  goal  to  illusfrate  the  number  feature  of 
data  elements  causes  two  goals  to  be  generated: 

GEVERATE-EXAMPLE-SIHGLE-ELEKEIIT 

GEItERATE-EXAMPLE-lfULTIPLE-ELEMERTS 

SO  as  to  highlight  the  difference  in  number  of  elements  be¬ 
tween  the  two  examples.  (Each  of  these  goals  posts  further 
goals  to  actually  refrieve  the  example  and  generate  an  appro¬ 
priate  prompt,  etc.)  The  example  generation  algorithm  en¬ 
sures  that  the  examples  selected  for  related  sub-goals  (such  as 
the  two  above)  differ  in  only  the  dimension  being  highlighted. 

The  goal  to  illustrate  the  type  dimension  using  examples  ex¬ 
pands  into  four  goals:  to  illusfrate  symbols,  numbers,  symbols 
and  numbers,  and  sub-lists  as  possible  types  of  data  elements. 
(The  other  combinations  possible  -  numbws  and  sub-lists, 
symbols  and  sub-lists,  and  all  three  together -are  also  possible 


*We  use  rhetorical  relations  from  Rhetorical  Structure  Theory 
(RST)  I  Mann  and  Thompson,  1987)  to  ensure  Ute  generation  of 
coherent  text  -  ELABORATE  is  one  of  the  relations  defined  in  RST 
that  can  connect  parts  of  a  text. 
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Figure  3:  Plan  skeleton  for  listing  the  main  features  of  a  LIST. 
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Figure  4;  Partial  text  plan  for  generating  the  LIST  examples. 


data  element  types,  but  are  not  illustrated  because  of  a  global 
constraint  that  the  number  of  examples  generated  should  not 
exceed  four  {[Dark,  1971])  and  the  system  attempts  to  present 
small  amounts  of  information  at  a  time  (because  of  the  user 
model).  Each  of  the  first  three  goals  posts  impropriate  goals 
to  retrieve  examples  and  generate  prompts.  The  first  goal  of 
generating  a  LIST  of  atoms  can  be  collapsed  with  the  previ¬ 
ously  satisfied  goal  of  generating  an  example  of  a  LIST  with 
multiple  elements  in  our  case. 

In  the  fourth  case,  the  user-model  prevents  the  system  from 
simply  generating  an  example  of  a  LIST  which  has  other 
LISTS  as  its  data  elements.  The  system  therefore  posts  two 
goals,  one  to  provide  background  information  (which  presents 
three  simple  lists),  and  the  other  to  build  a  list  from  these  three 
lists.  Heuristics  in  the  system  cause  the  system  to  generate  text 
for  this  fourth  case,  rather  than  just  a  prompt.  A  skeleton  of 
the  second  half  of  the  complete  text  plan  is  shown  in  Figure  4. 

The  resulting  discourse  structure  is  then  processed  to  make 


final  decisions,  such  as  the  choice  of  lexical  items.  Finally,  the 
completed  discourse  tree  is  passed  to  a  a  system  that  converts 
the  ZIFORN  goals  into  an  intermediate  form  that  is  accessible 
to  Penman,  which  generates  the  desired  English  output. 

5  Conclusions  and  Future  Work 

This  paper  has  largely  focused  on  the  issues  that  need  to  be 
addressed  for  an  effective  presentation  of  information  in  the 
form  of  a  description  with  accompanying  examples.  We  have 
identified  and  outlined  the  various  questions  that  need  to  be 
considered,  and  shown  through  the  use  of  examples  in  the 
domain  of  LISP,  how  some  of  these  may  be  computationally 
implemented.  We  have  integrated  a  text  generator  with  an 
example  generator,  and  have  shown  how  this  framework  can 
be  used  in  the  generation  of  coherent  and  effective  descrip¬ 
tions.  Our  work  is  based  on  an  analysis  of  actual  instructional 
materials  and  books  and  other  studies  on  examples.  It  illus¬ 
trates  the  possibility  of  planning  and  generating  examples  to 
illustrate  definitions  and  explanations,  by  considering  both  of 
them  together  during  the  planning  operation.  This  method 
also  takes  into  account  various  linguistic  theories  on  the  order 
of  presentation  of  facts  in  the  descriptions,  and  extends  them 
to  the  presentation  of  accompanying  examples.  Our  system 
recognizes  the  importance  of  information  that  is  usually  im¬ 
plicit  in  the  sequence  order,  and  maintains  information  in  the 
discourse  tree  that  would  allow  it  to  generate  prompting  in¬ 
formation.  We  have  illustrated  our  framework  with  a  brief 
trace  of  an  actual  description  that  uses  examples.  Our  work 
expands  on  previous  work  that  has  been  limited  to  the  inclu¬ 
sion  of  examples  with  text  -  without  explicit  regard  for  many 
of  these  factors  such  as  sequence,  content  of  each  example 
and  prompts. 

In  future  work,  we  shall  investigate  questions  on  issues  such 
as  when  an  example  should  be  generated,  and  how  a  previous 
one  may  be  easily  re-used  after  appropriate  modification. 
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